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Abstract 



m t 

J> | If the charged Higgs boson H + exists with m#+ < mt + m^, the conventional 

t> 

expectation is that it will decay dominantly into cs and t + v t . However, the 

o . 

three-body decay mode H + — > W + bb is also present and we show that it 
| becomes very important in the low tan (3 region for m H + > 140 GeV. We then 



Qh| explore its phenomenological implications for the charged-Higgs-boson search 

i- 

43 



Q , in top-quark decay. 



^ ■ The discovery of the top quark at the Tevatron collider has generated a good deal of 

current interest in the search for new particles in the decay of the top-quark. In particular, 
top quark decay is known to be a promising reaction to look for the charged Higgs boson of 
a two-scalar doublet model and, in particular, the minimal supersymmetric standard model 
(MSSM) 0. In the diagonal CKM matrix approximation the MSSM charged Higgs boson 
couplings to the fermions are given by 

L = —fJ- H + [cot/5 m ui Uid iL + tan/? m di Uid iR + tan/? m H i' i l iR ] + h.c. (1) 

V 2 mw 

where tan f3 is the ratio of the vacuum expectation values of the two scalar doublets and 
the index i labels the quark and lepton generation. This interaction implies a large H + tb 
Yukawa coupling when 

1 



tan/5^1 and tan j3>m t /mb (2) 

where one expects a large branching fraction for t — > bH + decay (given m% > m#+). In- 
terestingly the regions tan/3 ~ 1 and >m t /mft are favored by SUSY-GUT models for a 
related reason - i.e. the unification of the b and r masses which requires a large negative 
contribution from the top Yukawa coupling to the renormalization group equation 

It should be noted here that the perturbation theory limit on the H + th Yukawa coupling 
requires 

0.2 < tan/3 < 100 (3) 

while the GUT scale unification constraint implies stricter limits 

1 < tan/5 < nit/nib (4) 

which are also required if one assumes the perturbation theory limit on the Yukawa coupling 
to remain valid up to the GUT scale |J. Without any GUT scale Ansatz, however, the 
allowed region of tan/5 extends down to 0.2. We shall assume only the particle content 
of the MSSM Higgs sector but no constraints from GUT scale physics. Our analysis will 
remain valid in any two-Higgs doublet model satisfying the coupling pattern of the MSSM 
as given by ([TJ); i.e. the so-called class II models ||. 

For m t > itlh+ the dominant decay modes are usually assumed to be the two body decays 
H + — > cs, r + v. The corresponding widths are 

T cs = %P^r (ml cot 2 p + ml tan 2 /?) (5) 

^ = OCT 111 ? m T taI12 Z 3 ( 6 ) 

327m% 

The leading QCD correction is taken into account by substituting the quark mass parameters 
for eqs. (|IJ) and (|5|) by the running masses at the H + mass scale. Its most important effect 
is to reduce the charm quark mass m c from 1.5GeV to lGeV J/J. Consequently the two 
rates are approximately equal when tan/? ~ 1; the rv (cs) rate dominates when tan/5 > 1 
(tan/5 < 1). 



In this note we shall consider the phenomenological implications of a very important 
3-body decay channel of the Higgs boson, namely 

H + -> bbW + , (7) 

where the bW + comes from a virtual t quark ||. The dominant contribution comes from 
the top-quark exchange with a large Yukawa coupling of H + to the top quark given by the 
first term in eq. ([!]). One can easily calculate the corresponding width as 

d^ibw 1 ( 3g A mt cot 2 (3 \ r 2 / 2\ , / 2 sW 2 

, , = za 3 3 ,4/2 ^2 m w \ s w -2m b ) + [si -m b -m w ) [s b - m b - u, 

ds- b ds b 2567Hm^ + \4t7% (m 2 - s b ) J 1 v ' v /v 

(8) 

where s^, and sw are the 4-momentum squared transferred to the corresponding particles 
satisfying s b + s b + s w = m H + 2 + + 2ml @- 

Figure |l| compares the 3 body decay width T bbW with the 2 body widths T cs and T ri/ 
over the charged Higgs boson mass range 120 — 170GeV at tan/3 = 1. T bbW is seen to 
be the dominant decay width for m^+>140GeV, while the 2 body decays dominate up 
to m#+ = 130GeV. The reason for this is the large H + Yukawa coupling to tb, which is 
about 100 time larger than those to the cs and t + v channels. This can overcome the extra 
suppression factors due to the gauge coupling of the W as well as the 3 body phase space, 
provided the off-shell propagator suppression factor is not too large. The latter is ensured 
for m H + > 140GeV. Thus the 3 body decay ([?[) is the dominant mode for 

m H +^140GeV and tan/3<l (9) 

while the tu mode (|6]) dominates at larger tan (3. The cs mode is relatively small at all tan /3 
for m H + > 140GeV. It may be noted here that the relative size of the H + decay widths at 
tan p = 1 (Fig. [I]) would hold for all values of tan f3 in the two-Higgs doublet model of type 

10- 

This situation has a close parallel in the neutral scalar sector. For a neutral Higgs H° 
whose mass is slightly below the WW threshold a good detection channel is WW* with 



W* — > £za In this case the decay if — > VF£z/ is comparable to if — > 66 || . A related decay 
i/+ — ► W + Z* with Z* — > 66 is not considered because for multi-doublet models there is no 



H+W~Z coupling PJ. 
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FIG. 1. Comparison of the 3 body decay width T H+ ^i bW (solid) with the 2 body widths 
T H +^, C g (dashed) and r 'h+~>t+u (dots). 

The if ± search strategies in top quark decay have so far been based on the distinctive 
features of the channels 



t -> bH + -> 6r+z/ 
t -> bH + -> bcs 



(10) 
(11) 



vis-a-vis the standard model decay 

t -> 6W^+ -> 6 (&/, rz/, g'g) (12) 
As we have seen above, however, this strategy is valid only up to ~ 130GeV. For 



rriH+ ^ 140GeV the cs mode (|TT| ) is overtaken by 

t 6#+ -> 666M/+ -»■ 666(*V, rz/, q'q) 



(13) 



as the dominant decay mode for the low tan/5 (^,1) region. The distinctive feature of this 
new channel is evidently very different from those of the channels fllQf ) and ( |TT|). 

In order to assess the impact of the new channel ([13]) let us summarize the main features 
of the current H + search program in tt decay. It is based on two strategies — i) excess 
of tt events in the r channel, and ii) their deficit in the leptonic [I = e, fi) channel with 



respect to the standard model prediction from (|i2|) . The first is appropriate for the large 
tan p region where the rv channel ([!(]) is the dominant channel of the charged Higgs decay. 
One can already get significant limits on m#+ for very large tan (3 ( ^mt/mb) from the CDF 



tt data in the £t and inclusive r channels [ 1 1 , 1 2 1 . This analysis can be extended down to 



lower values of tan j3 at the Tevatron upgrade and the LHC by exploiting the opposite states 
of r polarization from W ± and decays ||13|| . Evidently this type of analysis would not 



be affected by the new channel. 

The second strategy is based on a suppression of the leptonic (e, /i) decay of the top due 
to the H + channels (|T0D and fllT| ) []. The experimental estimate of the tt cross section is 
based on the U and £+multijet channels with a b— tag, requiring leptonic decay of at least 
one of the top quarks. Thus the presence of the H + channels ([II]) and (|T1|) would imply a 
decrease of this ti cross section, while the experimental estimate 



MCDF+D0) = 6.5 ±i;| pb a if (CDF) = 7.6 ±J;1 pb (14) 



is actually slightly higher than the QCD prediction of <j t t < 5.6pb fi~5| . This has lead to 



a significant lower limit on m#+ at low tan/? (^/l), assuming dominance of the cs decay 
channel ([XT] ) |P^B ) P^[ - Evidently this method will be valid only up to m H + = 130GeV. Beyond 



this value the dominant charged Higgs decay channel in the low tan/5 (^,1) region is (|13|). 
which does not imply any reduction in the leptonic decay of the top. Instead it implies 



1 This is evident for the cs channel (|Tl]) but should also hold for the rv channel (10) as well since 
the e, fj, from r decay are expected to be soft and hence suppressed by the px cut used in the 
analysis. 



an increase in the b— tagging efficiency due to the multi— b final state. Since the CDF cross 
section is largely based on the b— tagged events, the presence of the decay channel ( [T3D would 
imply an increase of this cross section relative to the standard model prediction, instead of 
a decrease. Thus it will go in the same direction as the data. 

Let us now look at the implications of the new if ± decay channel ( |T5D on Tevatron tt 
events more closely. In Fig. § we show the branching fractions for t — > bH + and H + — > bbW 
decays over the low tan/3 region for = 140 and 150GeV. Also shown is the product of 
these two branching fractions, 

B — B(t — ► bbbW) = B(t -> bH + )B(H + -> bbW) (15) 

which is about the same for both values of . We see that this branching fraction lies in 
the range 5 — 20% for tan (5 = 1 — 0.6. This corresponds to a probability of about 10 — 40% 
(~ 2B) for the channel 

tt^bbbbWW (16) 

where one of the top-quarks decays via an H and we have made a first-order approximation 
in B. Thus the 2b and 4b final states occur with relative probabilities 1 — 2B and 2B 
respectively, where the former also includes a small contribution from the 2-body decays of 
the ff ± .. 

It should be mentioned here that the decay of the H ± into a neutral Higgs and a real 
or virtual W boson is (whenever kinematically allowed) an additional source for a 46 final 
state such as (|l^). Within the MSSM this contribution can be significant over the low tan/3 
region || depending on the SUSY breaking parameters. Thus the 3-body decay considered 
above constitutes a minimal contribution to the 46 final state (|i~6"D generated by the decays 
of the charged Higgs boson. 

We have studied the characteristic features of the above channel versus the standard 
model decay 

tt -> bbWW (17) 
6 
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FIG. 2. Branching fractions for t — > bH + (dashed lines) and H + — * bbW (solid lines) decays for 
low tan p. Heavy lines and thin lines correspond to = 140 and mjj+ = 150GeV respectively. 
The dotted line corresponds to the product B(t — > bH + )B(H + — * bbW) for m#+ = 140GeV (the 
plot for rriff+ = 150GeV is practically identical). 

via a parton-level Monte Carlo program. While the I and v from W decay have very similar 
kinematic distributions in the two cases, there is a clear difference in the number of tagable 
5-quarks. The CDF SVX detector has a tagging efficiency of e& = 0.24 per b satisfying 



E b T > 20GeV, | Tfc | < 2 



(18) 



which takes into account the loss of efficiency due to the limited rapidity coverage of the 



vertex detector (\r] svx \<l) This is expected to go up to e& = 0.4 per b for run II as 

the rapidity coverage of the vertex detector is extended to \r] svx \ = 2. Table 1 shows the 
probability distribution of the numbers of b quarks per event satisfying the tagging criterion 
( |l"8D for the signal (|TE| ) and the standard model background ( |P7D channels. It shows that the 
majority of the signal events are expected to contain 3-4 tagable b quarks for m#+ = 140GeV 
(similar results hold for m#+ = 150GeV). It also shows the probability distribution for the 
expected number of 6-tags per event for the SVX tagging efficiency of e& = 0.24, where we 



have assumed that the uncorrelated probability for tagging n out of N tagable 6-quarks is 
= rQe^ (1 — Eb) N ~ n . The corresponding expectations for the run II efficiency e& = 0.4 are 
shown in parenthesis. The implications for the ti events in the 6-tagged ^+multijet channel 
are discussed below. 



Probability (%) 
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. of taga 
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it -> bbbbWW (2B) 


4.7 


25.6 


50.6 


18.9 


52.8 (74.2) 


12.4 (31.8) 


(6.6) 


it -> bbWW (1 - 2B) 


13 


87 






39.6 (60.9) 


5 (13.4) 





Table 1. Probabilities for different numbers of tagable b quarks per event and numbers 
of 6-tags (per event) with e& = 0.24 (0.4) for the signal (m H + = 140GeV) 
and the standard model background. 

As we see from this table the probability of inclusive single (> 1) 6-tag is 52.8% for the 
signal compared to 39.6% for the standard model decay, i.e., about 1/3 higher. Consequently 
the measured it cross section will appear larger than the standard model prediction by 
(1/3) x (2B), i.e. about 13% for B = 0.2. This could account for at least part of the excess 
of the CDF ti cross section over the standard model prediction. Even more significantly, 
the probability for inclusive double (> 2) 6-tag is 12.4% for the signal compared to only 5% 
for the standard model decay, i.e. an excess of 150%. This would imply an excess of double b 
tagged events over the standard model prediction by 3-B, i.e. 60% for B = 0.2. Again there 



seems to be an indication of such an excess in the CDF data ||19|| . It should be remarked 
however that the excess is expected to appear in the > 3 jet events; but not in the 2 jet 
sample, except through fluctuations. It is therefore premature to link the reported excess to 
the above mechanism. It is important to note, however, that the size of the signal can have 
visible impact even at the level of the existing limited data. 

It should be noted here that one expects a 20-fold rise in the number of ti events in 
the run II, and the efficiency of single and double tags to go up by factor of 1.5 and ~ 3 
respectively. Thus one expects about 1000 single (> 1) and 200 double (> 2) 6-tagged events 
for CDF, and similar numbers for D0 (in run II). Even with a B of only 5%, this would 

8 



correspond to an excess of ~ 30 double 6-tagged events, i.e. a 2 — 3a effect. Moreover, the 
6.6% efficiency for > 3 6-tags for the signal would imply at least 10 — 12 triple 6-tagged events 
for B > 5%. Finally, one should be able to get additional constraints from the clustering of 
the reconstructed if 1 * 1 mass. 

Thus the 3-body decay channel provides a visible signature for a charged Higgs boson in 
top-quark decay over its region of dominance, i.e. m H + > 140GeV and tan/3^,1. This can 
be used to probe for an i^T ± at the Tevatron run II over the mass range 140 — 150GeV, and 
can be extended beyond 160GeV at the LHC. We conclude with the hope that this channel 
will play an important role in the charged Higgs boson search program in the future. 
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